Repository navigation
Conversation
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
06a6490 to
edc4312
Compare
6de072e to
2fadfee
Compare
jotitadev
left a comment
jotitadev
left a comment
There was a problem hiding this comment.
export_gap does not run on Python 3.9 (the repo's floor).
Machine: Apple M2 Max, macOS 27.2. Pythons: /usr/bin/python3 = CPython 3.9.6 (Xcode), and 3.14.7.
PR head: 2fadfee, run as python3 tools/tests/test_export_gap.py.
- CPython 3.14.7:
OK(27 tests). - CPython 3.9.6:
FAILED (errors=22), every one:
TypeError: write_text() got an unexpected keyword argument 'newline'
attools/export_gap.py:321and:323:destination.write_text("".join(nid + "\n" for nid in missing_nids), encoding="utf-8", newline="\n") ... destination.write_text("", encoding="utf-8", newline="\n")
Path.write_text's newline keyword was added in Python 3.10, so on 3.9 the tool raises before
writing anything. CI did not catch it because the runners use Python 3.12/3.13, not 3.9.
To keep the explicit-LF behavior on 3.9, write through open instead, e.g.:
with open(destination, "w", encoding="utf-8", newline="\n") as handle:
handle.write("".join(nid + "\n" for nid in missing_nids))Written with AI assistance (Claude Code). I ran these Python tests on my machine.
|
@juanpimg export_gap.py:321/:323 uses Path.write_text(..., newline=...), which requires Python 3.10, while the registered test still permits Python 3.9. Could you use Path.open/open with newline followed by write for both empty and nonempty output, then run the complete export_gap suite on Python 3.9 and a current Python version? |
THEMANnt
left a comment
THEMANnt
left a comment
There was a problem hiding this comment.
Reviewed 2fadfee89939 against author merge base 6f9de5738615. Requesting changes because the new gap report incorrectly treats unpatched helper names as providers of a hashed NID.
Do not count _nid_no_patch and _nid_no_patch_cut as hashed exports — tools/export_gap.py:124–128, 167–175. base_of strips all three suffixes, then collect_provided hashes the base name. However, the exact repository resolver in core/libs/nid/src/NidResolver.cpp:15–20,45–51 keeps _nid_no_patch names unchanged and strips _nid_no_patch_cut to a literal name without hashing. Only _nid_postfix has the hashed-base behavior. With a console table requesting T2IWNGg3cnI (sceExampleNoPatchBase) and a source defining only sceExampleNoPatchBase_nid_no_patch, the submitted Python tool reports 0 missing, 1 already provided; the cut variant produces the same false result. Thus the emitted stub list omits an export that remains unresolved. This is reproduced in disposable source/table/database fixtures; resolver semantics are verified directly from the pinned source. Please model the actual resolver and correct the test that currently expects both unpatched variants to satisfy this NID. Preserve explicit APS5_EXPORT bindings as valid providers.
Windows x64 / CPython 3.11: all 27 export-gap tests, 37 NID-name tests and 16 stub-generator tests pass. Supporting suites also pass on the author merge base. The new suite cannot run there because the new module does not exist; that import error is not regression-sensitivity evidence. Conventions, whitespace and registration pass. Hosted relinker matrix/ASan and conventions jobs succeed; full product jobs are skipped. An attempted standalone native resolver control did not link because its dependency set was incomplete; no successful native execution is claimed and those link errors are not author defects.
|
Windows run of this head
AI-assisted: yes (Claude Code) |
2fadfee to
176a618
Compare
|
Thanks — fixed. |
|
Thanks — fixed as requested. Only |
AnyPS5 resolves a title's imports by NID, so a title that imports a system function that no AnyPS5 library provides fails to load with an unresolved import. The pull requests that declare those exports build the same list by hand: the export table of the console's library, intersected with the pinned aerolib names, minus what the repository already defines. export_gap computes that list. It recognises what a module already provides in all four ways the project does it: an APS5_VABI declaration, a name ending in _nid_postfix, whose NID is the one of the base name, an APS5_EXPORT binding, and a symbol produced by a macro. It reports the NIDs the database does not carry, which the maintainer's rule leaves out, and the missing exports whose name the module calls itself, which must not be declared under that name because the declaration would interpose on the module's own call. The output is the NID file tools/stubgen.py takes.
The console's export table carries a C++ mangled symbol, _Z5dummyv, whose name cannot be declared as a C function. Listing it as an ordinary missing export would lead a contributor to declare it under that name, so it now goes in the category that says the export needs a renamed helper bound with APS5_EXPORT. A name that starts with an underscore but is not mangled stays an ordinary missing export, and the NID list stays complete, because tools/stubgen.py binds a NID it cannot declare under the real name with its UnknownNN helper. The module name check also accepts the libraries whose name ends in .native, which are real directories under core/libs/prx, and still rejects a name with a path separator.
… working _nid_no_patch resolves to itself and _nid_no_patch_cut to its stripped base; only _nid_postfix hashes. Write output files with open(..., newline) so Python 3.9 works.
176a618 to
c4563f1
Compare
|
Update: rebased onto current |
THEMANnt
left a comment
THEMANnt
left a comment
There was a problem hiding this comment.
Re-reviewed c4563f10df5a5900a0c2f210a112585e42f78de4. The earlier suffix-resolution finding is fixed: independent fixtures now match the pinned resolver for both literal no-patch variants, hashed _nid_postfix, and numeric disambiguators.
[P2] Protect export-table inputs from --out — tools/export_gap.py:322–324,349–354. The existing alias guard checks --nids and --db, but omits the selected <export-table>/<module>.c files. Running --module libSceExample --export-table tables --db names.txt --out tables/libSceExample.c reads a valid table, then opens that same input with "w", replaces its contents with T2IWNGg3cnI\n, and exits 0. This silently destroys the export inventory. The same fixture with a separate output preserves the table; the existing database-alias guard correctly refuses its analogous case.
Please include every selected export-table file in the input/output alias validation before writing, including resolved-path and hardlink aliases, and add preservation regressions.
Windows/Python 3.11: export-gap 36/36, NID-name 45/45 and stub-generator 23/23 passed. The independent reproduction runs a byte-identical copy of the submitted CLI in a disposable fixture. The hosted Windows relinker job merged this head and passed export_gap; that suite does not cover this alias. Initial supporting-suite sparse-checkout/sandbox failures were corrected; no product defect is inferred from them. No game execution was performed.
|
Static review plus an independent Linux run of the full Python test module (CachyOS, Python 3.14) and What I checked (static)
Test run
Coverage note 36 tests in 0.017s suggests table-driven fixtures over the parser/intersector (likely one per input-corruption class) rather than an end-to-end run against a real library's export table. That is the right unit of testing for a report-only tool; a real |
|
While scoping the remaining Examples in the tree: Fixed in This does not change the output for modules without assembly definitions. |
…e_diff The registration sat directly after the annotate_module block. PR boykopovar#8588 adds export_gap at the same line, so the two pull requests would conflict textually. The test has no other dependency, so it moves to a position that no open pull request's CMake hunk touches.
What
Adds
tools/export_gap.py, which reports the system library exports that AnyPS5 does not provide.AnyPS5 resolves a title's imports by NID, so a title that imports a system function that no AnyPS5 library provides fails to load with an unresolved import. The pull requests that declare those exports — #8021, #8028, #8330, #8331, #8333, #7795, #8240, #8328, #6584, #8255 — all build the same list by hand:
The tool computes that list and writes it in the format
tools/stubgen.py --nidstakes, so the whole job becomes two commands:It recognises what a module already provides in all five ways this project does it: an
APS5_VABIdeclaration, a name ending in_nid_postfix(whose NID is the one of the base name), anAPS5_EXPORTbinding, a symbol produced by a macro, and a definition emitted byAPS5_ASM_FUNCTIONorAPS5_SIGNAL_ASM_FUNCTION.Two things it gets right that a hand-built list does not, and that cost real pull requests while this tool was being written:
libSceVoiceChatdeclares sixteen of them throughVOICECHAT_OFFLINE_REQUEST(NAME). A text scan ofAPS5_VABIdoes not see them, so the list counts them as missing and a pull request that declares them again exports the same NID twice. The tool reports all sixteen as already provided.core/libs/prx/libScePosixForWebKit/Resolver.cpp:186calls the host's::gethostbyname_r, so declaringgethostbyname_rin that library broke its resolver andguest_resolverfailed with "gethostbyname_r not implemented". The tool reports that export as needing a renamed helper bound withAPS5_EXPORT, the waylibSceAmprre-exports the libc functions, with the file and the line of the call.It also reports the NIDs the database does not carry, which the maintainer's rule leaves out of the list.
Tested
CachyOS Linux, kernel 7.2.9-2-cachyos, Python 3.14.8, Release build.
export_gap: 39 test cases, all passing, no network access. The cases cover a module whose exports are all missing, a module that provides an export under each of the three postfixes, a module that provides an export through a macro, a module that emits a definition withAPS5_ASM_FUNCTION(with and without anAPS5_EXPORTalias), a module that binds a NID explicitly withAPS5_EXPORT, a NID the database does not carry, a missing export the module calls itself, the same result from the--nidsinput as from the--export-tableinput, and the error paths.python3 tools/check_conventions.py --base origin/main: all conventions checks passed.cmake --build build --parallel: no error.ctest --test-dir build -R export_gap --output-on-failure: 1/1 passed.libSceVoiceChatreports0 missing, 30 already provided, and names the sixteen macro-produced symbols as provided through the macro.libScePosixForWebKitreports24 missing, 12 already providedand namesgethostbyname_ras called atcore/libs/prx/libScePosixForWebKit/Resolver.cpp:186.No title was run: the contributor has no PS5 and no game dumps.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)